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ABSTRACT 

This annotated list of Educational Testing Service 
(ETS) studies includes the following types of investigations with 
findings related to low socioeconomic and minority groups: studies 
designed specifically to investigate the educational, personal, and 
social characteristics of the disadvantaged; studies of other 
populations of minority groups as a variable; studies concerned with 
the development of theories, methods, and techniques which hold 
promise for more effective analyses of data collected in connection 
with studies of the disadvantaged; and surveys of literature and 
analyses of theoretical concepts which have significance for research 
on the disad vantagedi The main list contains resumes of 76 studies 
completed from 1951 to August 1969- These are listed in 11 
categories; background factors (7 studies) ; cognitive processes, 
development and functioning (8) ; educational and vocational 
attainment (6) ; evaluation of programs (17) ; literature reviews (6) ; 
prediction (12) ; racial attitudes (2) ; school characteristics (2) ; 
test characteristics (2) ;• test performance (3) ; theories, methods, 
and techniques (9). Also listed with brief descriptive annotations 
axe 22 studies in progress. Availability of reports from ETS is 
indicated, and citations are supplied for the 18 studies published as 
journal articles. (JS) 
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INTRODUCTION 



This annotated list of ETS research studies includes the 
following types of investigations which have yielded findings 
relevant to low socioeconomic and minority groups (i. e., the 
disadvantaged) : 

Studies designed specifically to investigate 
the educational, personal, and social 
characteristics of the disadvantaged 

Studies of other populations or characteristics 
in ^ihich socioeconomic status or the 
performance of minority groups are a 
variable 

Studies concerned with the development of 
theories, methods, and techniques which 
hold promise for more effective analyses 
of data collected in connection with 
studies of the disadvantaged 

Surveys of literature and analyses of 

theoretical concepts which have significance ‘ 
for research on the disadvantaged 

The list includes studies, or phases of studies, completed 
during the period 1951 to August 1969, as well as studies in 
progress. No reports are available for the latter. Single 
copies of the former are available as long as the supply lasts. 
In some instances, these reports have been published as journal 
articles. Such reports have an asterisk after the author's 
name, and the full reference to the journal article is given 
in the Appendix. Occasionally, the titles of journal articles 
vary slightly from the titles given in the ETS documents. 



COMPLETED STUDIES 



BACKGROUND 

FACTORS 



0 



Adolescent Behavior and Interests 

The Background and Experience Questionnaire (BEQ) was 
constructed to assess aspects of cultural orientation of 
adolescents. It covers three types of behavior: TV viewing, 

reading, and other extracurricular activities, at high, medium, 
and low levels of cognitive demand. Items were selected to 
reflect the preference of students for adult or teenage culture. 
This report analyzes the 1963 data on scores for a subsample of 
268 boys and 210 girls from a total group of 32 public and 
private secondary schools In the ETS Growth Study. 

Some of the findings were that as a group, adolescents 
were not oriented toward one culture while rejecting the other. 
Socioeconomic status and academic aptitude were correlated 
positively with high-level scores and negatively with low-level 
scores. The BEQ appears to have promise as an Instrument to 
aid In counseling. 



M. H. Maler and Scarvla B. Anderson 
RB 64-52, CEEB RDR 64-5, No. 6 



Cultural Deprivation or Whatever It Is 

After outlining the Methodology for A Study of Academic 
Prediction and Growth , the author discusses the problem of 
defining cultural deprivation, citing such authorities as 
Rlessman, Weaver, Anastasl, Passow, etc. For the ETS Growth 
Study, cultural disadvantage Is defined for certain purposes as 
a principal's estimate on a seven-point scale of average parental 
education level In a community; for other purposes. It Is defined 
In terms of student responses to a questionnaire. 

There was a strong association between parents' SES and 
test performance of students. Cultural advantage did not 
Improve prediction when students were In the last two years of 
college-preparatory high school programs. 

There Is heed for more precise definitions of terms and for 
additional measures of cognitive and noncognltlve development. 
Poverty does not necessarily mean cultural deprivation, and 
counselors must be aware of the multivariate nature of cultural 
phenomena. Local norms are Important In this respect. 

T. L. Hilton 

Paper presented at a symposium on 
Implications of Research In Cultural 
Disadvantage for Coiinsellng Psychology , 

APA 1964. 
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BACKGROUND 
■ FACTORS 
(continued) 



Educational Achievement Pylor to and During High School 

A study supported by a grant from the U.S. Office of Education 
and based on ETS Growth Study data* To examine the argument that 
many achievement differences among students are associated more 
with SES than with curriculum or enrollment, multivariate analysis 
of variance was ‘carried out on 7th-grade STEP and SCAT scores and 
SES scores (the latter based on selected questions from the Background 
and Experience Questionnaire) . Major analysis sought to establish 
a baseline with grade 7 data and an indication of change with grades 
9 and 11 data. 

In general, lower SES groups seemed to perform at a higher Icivel 
of achievement on more specific (e.g., STEP Science, STEP Listening) 
than general attributes, and some lower SES schools made greater 
gains in achievement over time. Nonacademic groups had lower and 
more uneven achievement performance than academic groups, which 
implies a need for rethinking the kind of education noncoil egebound 
students should receive. 

M. S. Patton 
RB 69-29 



The Effects of Age , Sex , and Socioeconomic C la ss on Responsiveness 
to Two Classes of Verbal Reinforcement 

One of the two experlm<^nts reported is a replication of 
previous research which indicated that lower-class children are 
more responsive to person reinforcers (i.e., "good," "fine") while 
middle— class children are influenced more by performance reinforcers 
(l.e., "right," "correct"). The subjects in Experiment I were 
20 middle-class and 40 lower-class children, drawn from the second 
grade of four public schools. Half of the lower-class children 
were Negro and within each class and racial group, half of the 
subjects were boys and half were girls. Experiment II was designed 
to examine the hypothesis that older children are more influenced 
by abstract reinforcers than younger children. The subjects in 
this experiment were 20 second-grade and 20 sixth grade— middle 
class children, equally divided among boys and girls in each grade 
drawn predominantly from one middle-class school. 

The findings of Experiment I were not replicated. As far as 
socioeconomic status is concerned, in Experiment I, middle-class 
girls and lower-class boys were more responsive to verbal 
reinforcement than middle-class boys or lower class girls, while 
within the lower class, boys were more responsive than girls to 
verbal reinforcement. In Experiment II, middle-class boys were 
more responsive than girls to person reinforcers, while girls were 
more influenced by performance reinforcers. In Experiment II, 
older children appeared to be more responsive than younger children 
to performance reinforcers, while boys in general appeared more 
sensitive to person, and girls to performance reinforcers. 

D. Rosenhan and Jean A. Greenwald* 

RB 64-39 
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T h e Effects of Social Class and Race on Responsiveness to 
Approval and Disapproval , 

This study examined the effects of approval or disapproval 
on the performance of lower- and middle-class children by 
simulating In the laboratory the conditions that may have 
originally produced their attitudes and behavior In the real world. 
The subjects were 72 first-grade boys drawn from two public schools 
of mixed socioeconomic class, which was determined on the basis 
of parental occupation. Twenty-four boys were middle- and 48 
were lower-class children. Half of the lower class were Negro and 
half were white. 

Lower-class children were more influenced by approval and 
disapproval than were middle-class children. Relative to middle- 
class children, the performance of lower-class children was 
markedly Improved by approval and markedly worsened by disapproval. 
While the overall effect of approval was greater than that of 
disapproval, lower-class Negroes performed better with approval 
and worse with disapproval than did middle-class whites. Also, 
the amount of Improvement for Negroes in the approval condition 
was substantially greater than it was for middle-class whites. 

D. L. Rosenhan* 

RB 65-38 



BACKGROUND 

FACTORS 

(continued) 



Financial Aid Awards of High School Seniors in the Northeastern 
States as ai Function of Socioeconomic Class 

An analysis of the relationship between socioeconomic status 
of financial aid applicants and college practices In distributing 
aid. The study, limited to the Northeastern area of the U. S., 
where the Scholastic Aptitude Test Is largely used for college 
admission aind financial aid decisions. Identifies five socio- 
economic classes; groups colleges according to source of control, 
type of student body, and type of Institution; and classifies 
awards In seven combinations of grant, loan, and job. 

Results show that In some ways upper-class students receive 
preferential treatment, apparently because of superior test 
scores. Lower-class applicants have a better chance of receiving 
aid than their upper-class peers, but awards to lower-class 
students more often consist of loans and jobs rather than the 
outright grants which go to the upper-class students. 

G. Schlekat* 

RB 68-10 



BACKGROUND 

FACTORS 

(continued) 



Talent Losses Before High School ; The Number of Tenth Graders 
In Various Aptitude and Social Categories Who Leave School 
Before Graduation 

A study restricted to what happens during the last five 
semesters of high school, to determine what percent of dropout Is 
concerned with able students* An academic aptitude test and a 
questionnaire were given to 9,700 public high school sophomores 
In 1955. In 1959, It was determined whether 9,500 of these 
students had graduated or not. Seventy-nine percent had graduated. 
Classified according to academic aptitude, 91% of the most able 
third graduated; about 80% of the middle third; and about 69% 
of the least able third. In the most able 10% who do not 
graduate, one student out of 30 Is doing falling work, and drops 
out before graduation. Dropout Is not random, but Is closely 
associated with such environmental factors as family characteristics 
school, community, and regional characteristics; peer group 
orientation; and the student's own self— picture. 

G. Stlce 

RM 60-01 

Paper presented at American Association 

for the Advancement of Science, Chicago, 1959 



COGNITIVE 

PROCESSES, 

DEVELOPMENT, 

FUNCTIONING 



I 

Let*s Look at First Graders 

AND Originally designed and produced by Educational Testing 

Service for the Board of Education of the City of New York, 
the first phase In this series of materials Is designed to 
help first-grade teachers understand, assess, and foster tha 
Intellectual growth of young children. 

The materials , Including the guide , Let*s Look at First 
Graders , are available from Cooperative Tests and Services, ETS. 

The second phase cf this project Is described In Studies 
In Progress under Let*s Look at Children . 



Cognitive Growth In Preschool Children 

As part of a three-year project being conducted by ETS 
In cooperation with the New York City Board of Education, 
with support from the Carnegie Corporation, this study was 
made to gain Information about the course of cognitive growth 
In the preoperatlonal or preloglcal child (age 4-6). Subjects 
were 259 children from four public schools and two private 
nursery schools, representing a range from low to middle SES. 
Areas In which tests were constructed and some findings are 
as follows: 

Classification Skills: There were clear developmental 

differences between age groups for low SES children. These 
differences were less consistent for the middle SES children, 
but there was some confounding of this result because the 
middle SES prekindergarteners had a considerably higher mean 
Peabody Picture Vocabulary Test IQ score than the middle 
SES children. 

Time Sequence Tasks: SES differences were observed 

In all four types of Items designed for the study — 1. e. , 
following linguistic cues, verbal recall and transfer, 
disposition of structure, and sequence completion. 

Distance, Movement, Number, and Time: There were 

significant differences In performance by age and social 
class but not by sex. 

Number Conservation: Middle SES children performed 

considerably better than low SES children. 
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GOGIIITXVE 

noctssis, 

DEVELOFianiT, 

rUMCTKNIZWC 

(coatiaiMd) 



R. S. Melton, Rosalind Charlesworth , 
Barbara Rothenberg, Masako N. Tanaka, 

L. W. Pike, and Anne M. Bussls 
RM 68-13 

Negro-White Differences In Adolescent Educational Growth 

The present stu^ was designed to compare, longitudinally, 
the ecedealc growth of Negro and white students who attended 
the seme high schools and were enrolled In the same curricula, 
end also to Investigate the relationship of a measure of 
socioeconomic status to academic growth. 

The results confirmed the findings of cross-sectional 
studies of differences In Negro-white academic growth: on 

tests, but not all, the Negro students exhibited a 
lesser degree of growth. The results also Indicated that 
tete of growth depended on which school system and which 
curriculum the students were enrolled In. 

A check on the wlthln-group homogeneity of regression 
indicsted that the regression slopes differed significantly 
smoag the groups categorized by race, sex, school, and 
cutriculum, and thus, conventional covariance adjustment 
for differences in socioeconomic status was not appropriate. 

The authors concluded greater effort should be made 
in collecting data about the schools and their subjects 
la order to gala a better understanding of circumstances 
and conditions that helped produce the observed results . 



Basic Language Structure: Most of the children's 

grammatical errors appeared to be governed by rules and 
AND strategies; they were not mistakes of random guessing. 

Lower-class children lagged significantly behind middle SES 
in understandings of same/different distinctions and in 
concepts of more/less, and they had difficulty with 
comprehension of plurals and prepositions. Low SES subjects 
performed comparably to middle SES children, however, in 
the co^>rehenslon of negation and conjuntlons. 



M. Rosenfeld and T. L. Hilton 
RB 69-71 



COGNITIVE 

PROCESSES, 

DEVELOPMENT, 

FUNCTIONING 

(continued) 



Negro-White Differences In Intellectual Growth 

and This paper reports on a longitudinal investigation of the 

Intellectual growth of 316 Negro and 501 white students In grades 
5, 7, 9, and 11 In one Midwestern and one Western City. As 
participants In the ETS Growth Study, all students had taken the 
Sequential Tests of Educational Progress, School and College 
Ability Tests, and a measure of socioeconomic status taken from 
the Background and Experience Questionnaire. Multivariate 
analyses of variance were used to analyze the data. 

Test scores of white subjects were significantly higher than 
Negroes at grade 5 on all tests. Analyses of covariance used to 
equate on racial differences in SCAT and STEP Indicated that 
whites not only start out higher but continue to grow faster than 
Negroes on most tests in the battery. Both Negro and white 
subjects continued to gain and grow, however, on all SCAT and 
STEP tests and neither group declined from one testing to the 
next, regardless of the curriculum In which the students were 
enrolled. The Town-Curriculum-Race interactions should be 
explored further to determine why Negro academic students 
showed accelerated rates of growth on Reading and Listening 
relative to white nonacademic students .In one town but not in 
another. 



M. Rosenfeld 

Paper presented at APA, Division 17 
Symposium: Exploration In Vocational 

Development and Education 



Academic Growth In Pre domlnan t ly Negro and Predominant ly 
White Colleges 

Literature is briefly reviewed supporting the fact 
that Negro colleges as a group do not compare favorably 
with predominantly white Institutions in financial resources 
and faculty qualifications. There has, however, been 
little research on the effects of the predominantly Negro 
college on Its students. This study Investigated the 
relative effectiveness of the two types of colleges. 

Seven predominantly white colleges were matched 
(according to purposes, type of control, and ability 
level of students) with seven predominantly Negro colleges, 
to compare student academic growth. The sample consisted 
of 406 students from the former and 327 from the latter. 

All had taken the Area tests of the Graduate Record 
Examination (GRE) Institutional Testing Program as college 
seniors and the Scholastic Aptitude Test (SAT) as entering 
freshmen. 



No significant differences were found between the 
two racial groups In GRE Social Science or Natural Science 
scores when SAT scores were used as covarlates. Inspection 
of regression lines within each of the 14 colleges also 
supported the conclusion that neither group of colleges 
was any more or less effective In student academic growth 
as measured by the GRE Area Tests. 

The results of this study provide no evidence that 
one group of colleges Is any more or less effective than 
the other In Improving the achievement of students who 
have similar aptitude test scores at the time of entrance 
to college. However, since neither group was completely 
representative of all colleges In Its racial category, 
generalizations from the sample In this study to colleges 
In either racial group are not warranted. 

J. A. Centra, R. L. Lliin, and' 

Mary Ellen Parry 
RB 69-39 



COGNITIVE 
PROCESSES , 
DEVELOPMENT, AND 
FUNCTIONING 
(continued) 



An Exploratory Study of Word Associations of Negro College 
Students 



This pilot study attempted to Identify and explore through 
the word association technique possible differences In 
association responses of Negro and white college students. With 
the use of group written procedures, two word association lists 
of 50 words were each administered to 50 Negro college students. 
Forty-one words were taken from the Kent-Rosanoff list, 29 from 
the Palermo- Jenkins list, and 30 were words used In analogy 
Items of the Scholastic Aptitude Test. 

The results were generally similar to those found In previous 
normative studies, although there were slightly smaller 
proportions of matching from class primary responses to noun, 
pronoun, and adverb stimulus words, and of opposite responses 
to "opposite-evoking stimuli." A number of the responses 
Indicated reading difficulty or misunderstanding of the word. 

L. H. Belcher and J. T. Campbell* 

?S 68-4 






COGNITIVE 

PROCESSES, 

DEVELOPMENT, 

FUNCTIONING 

(continued) 



Reasoning . Relationships . and Relatedness of Words ; A Study of 
the Effects of Sociocultural Level and Definitions on the 
Solution of Analogy Items 

This study attempted to determine 1) the effects of providing 
students with definitions In connection with analogy Items; 2) 
idiether providing definitions has differential effects on students 
of different sociocultural levels; and 3) where. In the solution 
process, differences occur In reasoning among students of different 
sociocultural backgrounds . 

Results showed 1) a significant effect attributable to 
occupational prestige level (e.g., the higher the level, the 
more correct answers) ; 2) a significance In facilitation of 
solutions Independent of occupational prestige level; and 3) 
significant differences among levels of occupational prestige 
In rated ease of relating word pairs. Subjects were llth-grade 
students. 



J. R. Gentile 
RB 67-36 



Toward an Experimental Analysis of Reasoning on the Scholastic 
Aptitude Test ; A Pilot Study 

To discover why the mean for Negro students seeking college 
admission is consistently lower than that of whites on tests such 
as the Scholastic Aptitude Tests, the technique of having subjects 
think aloud while solving antonym, analogy, and data sufficiency 
Items was used. The nine male volunteer subjects (eight Negroes, 
one white), ranging In age from 15 years, 3 months to 17 years, 

5 months, were enrolled In a morning school seeking to Improve 
the teaching and learning of "inner city," poverty-background 
high school students. Two groups of each type of Item were 
assembled and each ^ received one group of Items about which he 
was to think aloud, while the words from the other were used for 
free association. Instructions for antonym and analogy Items 
emphasized that the examiner was primarily interested In "what 
the Item makes you think about when trying to answer It." 

Subjects In the D (Definitions) treatment group were given 
definitions of words they did not know; subjects In the A 
(Answer) treatment groups were given the answers to each 
question. Instructions for the word associations were designed 
to elicit as many associates to the stimuli word as S could 
produce. 
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COGNITIVE 
PROCESSES, 
DEVELOPMENT, AND 
FUNCTIONING 
(continued) 

r 


The results Indicated that most of the subjects were 
quite verbally restricted, in the sense that they seldom spoke 
spontaneously. Also, they appeared to have few associates to 
words employed and many "no response," clang associates, and 
associates to words mistaken for other words. Providing 
definitions of words did appear to improve the quality of 
responses and to increase the probability of a correct solution. 
On the other hand, providing answers to questions did not appear 
to improve the quality of verbalizations. 


[ 


J. R. Gentile 
RM 66-26 
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EDUCATIONAL Adjustment to College ; A Study of 10 , 000 Veterans and Nonveteran 

AND VOCATIONAL Students In 16 American Colleges 

ATTAINMENT 

A study originating with CEEB with support from the 
Carnegie Corporation, and completed In 1951 by ETS. Some 
findings: veterans performed better than nonveterans of equal 

scholastic ability. Veterans \dio would not have attended 
college without GI Bill financial assistance performed better 
academically than did veteran classmates who would have gone to 
college anyway. This finding was expected to exert a favorable 
Influence on efforts to obtain Increased aid for needy students. 

N. Frederlksen and W. B. Schrader 
(Hardbound book, limited offset edition, 

ETS 1951) 

Changes In Nonwhite Employment 1960-1966 

This study compares changes in white and non-white employ- 
ment during 1960-63 and 1963-66, to determine the effect on 
employment of the Civil Rights Revolution. The data employed 
were percentage breakdowns of occupational distribution by race 
and sex which are published each month by the Bureau of Labor 
Statistics, and are based on a sample survey taken each month 
covering the week which contains the 12th of the month. 

The employment status of Negroes and other nonwhites 
presents a pattern of general Improvement during the period 
1960-66. The number of nonwhites In higher status occupations 
Increased substantially during this period. The proportion 
of nonwhites In the lower status declined, even though total 
employment showed a substantial Increase. A difference In 
rate of change was also noted for professional and managerial 
occupations, on the one hand, and the clerical, craftsmen, 
foremen, and operatives occupations on the other. However, 

"the employment gap" between whites and nonwhites is still 
extremely large, insofar as the better occupational 
categories are concerned. 

J. T. Campbell and L. H. Belcher* 

RB 66-53 

Equality of Educational Opportunity 

A survey conducted In response to Section 402 of the Civil 
Rights Act of 1964, concerning "lack of availability of equal 
educational opportunities for Individuals by reason of race, 
color, religion, or national origin, in public educational 
Institutions. . ." 



ERIC 
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EDUCATIONAL 
AND VOCATIONAL 
ATTAINMENT 
(continued) 



Achievement tests were administered In grades 1, 3, 6, 9, 
and 12. With some exceptions (notably Orlental-Amerlcans) , the 
average minority pupils' scores were as much as one standard 
deviation below the majority pupils' scores In Grade 1. At 
Grade 12, minority scores were farther below the majority than 
were the first graders'. 

Variations In facilities and curricula of schools accounted 
for less variation In pupil achievement as measured by 
standardized tests, than the quality of teachers. Pupils' 
achievement was strongly related to educational background and 
aspirations of their fellow students. 

A publication of the National Center for Educational 
Statistics, United States Department of Health, Education, and 
Welfare, Office of Education. 

U. S. Government Printing Office, 1966. 

J. S. Coleman and others 



Educational Testing Service was the contractor for the major 
public school survey, providing technical assistance In planning 
the survey , furnishing existing published tests for use In the 
survey, carrying out the administration of these tests and of 
special questionnaires developed by the National Center for 
Educational Statistics of the USOE, aiuilyzlng the data, and 
preparing the report. 



High School Attrition 

A study to describe the high school dropout with regard to 
educational, vocational, and sociological characteristics. Data 
were collected from a stratified sample of 9,699 tenth-grade 
students In 97 public schools. A follow— up was made two years 
later to see which pupils had completed high school. 

^School achievement was the most Important factor related 
to attrition. Nearly half of the pupils In the lowest quartlle 
or quintile of their class did not complete high school, and 
nearly half of those pupils who were over-age for their grade 
probably because of earlier school failures, did not graduate. 
Other Important factors were post-high school plans of students 
and their friends, and low academic aptitude (which did not bear 
a strong relationship to poor achievement) . In addition, family 
factors, nonacademic curricula, and regional factors (l.e., 
dropout more marked In the South than In other regions) emerged. 
There Is no single factor that can be pointed to as the cause 
of dropout. 




G. Stlce and Ruth B. Ekstrom 
RB 64-53 
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EDUCATIONAL Learning and Tea ching In Atlanta Public Schools 

AND VOCATIONAL 

ATTAINMENT A study undertaken by ETS In 1955-56 at the request of the 

(continued) Atlanta Board of Education , to assess and report differences In 

achievement between white and Negro students, differences between 
white and Negro teachers In efficiency, and to suggest policies 
for placement of students and for teachers In the event of changing 
conditions (l.e., desegregation). All pupils In grades 4 and 6 
were given the Stanford Achievement Test In reading and' arithmetic; 
®11 8th-graders took the Iowa Every-Pupll Test In arithmetic, and 
all 12th-graders took the Essential High School Content Battery 
(World Book) . Teachers took the NTE Common Examination, with 
primary teachers electing to take the NTE Optional Examination 
In Early Childhood Education. Results are summarized In Part I; 
charts, tables, and figures discussed In Part I are contained In 
Part II. 

In general, white pupils came close to national average 
In reading, with Negro students falling below the average. In 
arithmetic, the results were similar except that Negro students 
did slightly better than In reading. Both groups at the high 
school level did better In science and were poorer In social 
studies and In 12th-grade English — a possible reflection of the 
achievement of their teachers In these subjects as revealed by 
teachers' NTE scores. 



W. G. Findley 
SR 57-001 and 57-002 



Two Variables Associated With Differential Productive Cultural 
Involvement Among Lower-Class Negro and Caucasian Young Men 

The major hypothesis of this study was that. Irrespective 
of race, the level of economic and educational Involvement among 
lower-class young men was related to their achievement orientation 
and their sense, of the locus of control over events In their 
lives. The subjects,— 30 Negroes and 30 Caucasians between the 
ages of 16 1/2 to 21 years— were placed In three groups: Stable 

Performers, who had a 2-year history of consistent employment 
or school attendance; Rolling Stones, who had a history of 
consistently Interrupted employment or school attendance for 
2 years prior to the study; and Lookers, who had looked Into jobs 
and/or school during the past 2 years but never followed through 
on them. Intelligence test scores derived from the Revised Beta 
Examination, were held constant for all subjects and subjects 
were matched for socioeconomic status by using father's 
occupation. Four Individually administered tests measured 
achievement orientation and locus. 
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EDUCATIONAL 
AND VOCATIONAL 
ATTAINMENT 
(continued) 



As far as achievement orientation is concerned, an analysis 
of variance revealed that both degree of productive involvement 
and racial membership related significantly to need for achieve- 
ment; degree of productive Involvement effects were found on the 
expected occupational choice measure of level of aspiration, 
as well as a significant interaction effect of productive 
involvement and racial group membership; and productive involve- 
ment in the culture was found to relate significantly to all of 
the indices of risk-taking used in this investigation except 
the "risk score," but no significant racial differences were 
found on any of the measures of risk-taking. As far as locus 
of control is concerned, degree of productive involvement was 
related slgnflcantly to locus of control, l.e., the more 
involved subjects perceived themselves as being more in control 
of their environment than the opposite. 



W. S. Hall 
RB 69-7 
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EVALUATION OF Development of Evaluation Measures for Use with Neighborhood 

PROGRAMS Youth Corps Enrollees 

Report of the Initial phase In the development of a battery 
of 13 tests for research and evaluation with disadvantaged 
adolescents In a work-training program (U. S. Dept, of Labor 
Contract No. 66-00-09). The Introduction Includes Purpose of 
the Study, Description of the Neighborhood Youth Corps Program 
and Its goals and objectives, a discussion of tfeasurement 
Problems with Culturally Deprived Groups, Available Formal 
Measures, and Evaluation vs. Prediction. Chapter II deals with 
Development of the Preliminary Measures; Chapter III describes 
the Sample, Administration and Data Analyses. Chapter IV 
describes Item Analyses, Urban-Rural Comparison, Reliability and 
Validity, Intercorrelatlons and the Factor Pattern, Criterion 
Scales, and Specific Enrollee Measures. Chapter V deals with 
Conclusions and Recommendations. 

The 13 measures developed were In the areas of Job 
Knowledge (JK) ; Vocational Plans and Vocational Aspirations (VP 
and VA) ; Interest In Vocational Tasks (VI) ; Attitude Toward 
Authority (AA) ; Self-Esteem (SE) ; Deferred Gratification (DG) ; 

Job Seeking and Job Holding Skills (JS and JH) ; Motivation for 
Vocational Achievement (MVA) ; Practical Reasoning ; Map-Reading 
(PRM) ; Practical Reasoning: Zip-Coding (PRZ); Practical 

Reasoning: File-Card Sorting (PRZ); Enrollee Rating Scale (ERS) 

and Counselor and Supervisor Rating Scales. 

The most effective single measure In Intellectual-ability 
was Job Seeking Skills. Motivation for Vocational Achievement 
and Job Holding Skills were the most promising of the attltudlnal 
measures. Development of normative data on a larger and more 
diverse sample Is urged, along with development of alternate forms 
and post-training criterion measures of vocational adjustment. 

Phase II of this project Is described under Studies In Progress . 

N. E. Freeberg 
PR 68-5 



Evaluation of Integration of Evanston District 65 Schools : 

Interim Report 

An Interim report of a study undertaken by Educational 
Testing Service for the board of Education of the Co mmuni ty 
Consolidated School District No. 65, Cook County, Illinois, 
on the effects of school Integration by redlstrlctlng and 
busing, upon the pupils, teachers, and parents of the 
City of Evanston. (This study Is part of a three-year 
study funded for Evanston by the Rockefeller Foundation 
In March 1968.) 

1 
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EVALUATION OF 
PROGRAMS 
(continued) 

Work on this project Is continuing. See Studies In 
Progress for a description of current activities. 

D. P. Norton and J. Hsla 
PR 69-10 



Contents Include background for the study, designs. 
and procedures, data analysis (through April 1969 and 
projected analyses), and Information on reporting and 
costs. 



Evaluating Title Activities In the Chicago Public Schools 

A report prepared for the Chicago Board of Education. 
Chapter I: The Nature of Evaluation. Chapter II: Title 

I Decision-Making. Chapter III: General Evaluation (In- 

cludes objectives of the Chicago Title I projects, sug- 
gested evaluation design and Instruments, and considerations 
for data collection and analysis). Chapter IV: Specific 

Activity Evaluation. Requests for copies should be sent 
to Miss Elberta Prlutt, ESEA Director, Board of Education, 
228 North La Salle Street, Chicago, Illinois 60601. 

Work on this project Is continuing. See Studies In 
Progress for a description of current activities. 

G. L. Marco 
PR 68-6 



A Survey of Opinions Regarding 1967-68 Chicago ESEA Title i 
Progranifl 

A detailed report of the results of a questionnaire 
(developed by Educational Testing Service) distributed to 
key personnel In Chicago city schools and the community 
In June 1968. 

The questionnaire was designed to obtain Information 
about a) what the goals of Title I projects should be, 

b) how well Title I programs were accomplishing their goals, 

c) what programs should be discontinued or revised, and 

d) how programs should be revised. Ratings were obtained 
In five areas: arithmetic, attitude toward school, self- 
image, reading, and English skills. In addition an over- 
all rating was obtained. 
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Contents Include design of the questionnaire and a 
description of the respondents; ratings and reconnnendatlons ; 
and an .analysis of respondents' comments. The complete 
questionnaire Is reproduced In the Appendix. 

Requests for copies of the report should be addressed 
to Miss Elberta Prlutt, ESEA Director » Board of Education, 
228 North La Salle Street, Chicago, Illinois 60601. 

G. L. Marco and Sandra R. Landers 
PR 69-8 



An Evaluation of a Community Program of Compensatory Education 
for the Culturally Disadvantaged 

The Threshold Program of the Friends Neighborhood Guild In 
Philadelphia, an educational and counseling program to guide 
Negro high school students In obtaining acceptance Into post-high 
school education, was evaluated. The evaluation Included 
administration of ability, achievement, and Interest tests. 
Interviews with students, and a tabulation of high school 
performance. Comparisons with a control group Indicated that 
there was no change In program students with respect to 
traditional measures or number accepted In post-high school 
education. There was a significant change In the attitudes of 
students In the program and their parents, but this change did 
not appear to have a significant relation to post-high school 
acceptance. 



6. Lutz 1968 

(Study Is being offered as a doctoral 
dissertation at Lehigh University) 



Operation Upgrade : Effectiveness In Attaining Cognitive 

Objectives 

Operation Upgrade was a program of workshops conducted by 
the New York City Board of Education In District 8, for teachers 
of grades 5-8 from schools enrolling large numbers of minority 
group children. Its purpose was to Increase the teachers* 
knowledge of ethnic group characteristics, to change attitudes 
toward minority groups, and to construct effective curriculum 
materials for these children. ETS evaluated progress In the 
objective of teachers' knowledge of ethnic group characteristics 
and group relations. A 75-ltem test was administered after 15 
workshop sessions. 
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PROCKAMS 

(continued) 



Results of the test failed to support the effectiveness 
of the program. Apparently there was Insufficient stress In 
factual content In the workshop sessions. It Is not appropriate, 
however, to generalize from these results to other programs 
for teachers of other grades. Further Information may be 
obtained from Operation Upgrade, P. S. 188 Bronx, 1221 Spofford 
Avenue, Bronx, New York. 



T. S. Barrows 
Curriculum Studies Report 
June 1967 



Planning for Innovation ; A Case Study of New York City 
School-Community Interaction Umbrella Program 

This Is a stu^ of the nature of declslon-maklng within 
a functionally decentralized structure. In 1967 several school 
and community agencies submitted abstracts of projects they 
td.shed to undertake, to the Title III Office of the NYC Board 
of Education. The five "umbrellas” covering the programs were 
teacher growth^ curriculum; guidance— pupil personnel service; 
basic skills (e.g., English as a second language); and 
communication, with 10-15 projects under each umbrella. ETS 
was asked to evaluate the organizing and planning. 

Some participants were eager to protect the autonomy of 
their components, and were unwilling to grant Program Develop- 
ment Boards more than limited advisory and administrative 
roles; others would allow Boards to have greater decision- 
making responsibility. Communication among participants was 
a key variable; clear delineation of authority was a critical 
factor. Heterogeneity of backgrounds enhanced participants' 
contributions to solutions of joint problems. 

6. Gordon 
PR 68-1 



Personnel Services for the Model School Division 



The purpose of this report is to describe the organization 
and functions of the Department of Pupil Personnel Services 
(PPS) in the Washington, D. C. public schools in order to 
recommend to the Model School Division (MSD) which services 
its own decentralized organization. The MSD functions problems 
of public urban education. 

The survey was organized to describe PPS from two 
perspectives : that of the personnel in the divisions of the 

department, and that of the schools using the services of the 
department. Two major sources of data, interviews, and 
questionnaires were employed. Interviews were conducted 
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with the heads of the seven divisions that make up PPS. 
Questionnaires were circulated among the rest of the -PPS staff. 
Including the guidance counselors assigned to MSD schools, 
and among principals and assistant principals of those schools. 

Gita Wilder 

Curriculum Studies Report 



An Evaluation of ^ Summer Reading Institute , 1968 

The Model School Division (MSD) of the Washington, D. C. 
Public Schools, which functions to provide a testing ground 
for new approaches to the problems of public urban education, 
conducted a Summer Institute In Beginning Reading In Grades 
K-3. The objective of this Institute was to modify teachers' 
classroom behavior by providing training In four methods of 
reading Instruction. ETS assisted MSD personnel In the 
development of a) formal Institute objectives, b) a research 
design, and c) Instruments tailored to the specific needs of 
this program. 

A questionnaire, developed to assess many of the Institute's 
objectives, was administered to 69 K-3 teachers at the end of 
the six-week training session. Questionnaire data Indicated 
that the large majority of Institute participants said they 
learned "quite a bit" about reading methodology, found the 
content of the Institute relevant to their needs. Intend to 
Implement what they learned, and generally liked the way the 
Institute was conducted. 



M. Rosenfeld 
PR 69-3 



Bluffton Interterm ; Evaluation 

Bluff ton College conducts a short Interterm each year 
between Its first and second terms. During the 1968-69 
school year the chosen topic was poverty. The purpose of 
this study was to ascertain whether the program changed the 
students' attitudes towards and perceptions of poverty. A 
single stimulus version of the Allport, Vernon, Llndzey, 

Study of Values , Kaufman ' s Status -Concern Scale , and a number 
of Semantic Differential scales covering poverty related 
concepts were administered to the students. A random halves, 
pretest-posttest design was used. Changes In measured attitudes 
and perceptions were small although statistically significant 
In several cases. The data seemed to support a slight 
beneficial effect. 



T. S. Barrows 
PR 69-67 
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OF Project Head Start at Work 

Report of a survey of 1300 Head Start classes In 335 
Project Head Start centers during the summer of 1965 9 conducted 
for the Institute for Educational Development by Educational 
Testing Service and a team of 39 observers. Includes background 
and method of the study, samples and generalizations from the 
observation record, and Implications for education. Appendices 
Include the observation guide and an outline for summary of 
observations used by each observer, a map showing locations of 
the centers visited, and a list of centers by state. 

Among the outstanding observations Is the fact that preschool 
disadvantaged children have difficulty In communicating, and 
require an almost continuous Interaction with an older person — 
either an adult or a teen-ager — In the "teacher” role. 

J. E. Dobbin* 

(Report Issued by Institute for Educational 
Development, April 1966) 



Project Head Start Summer 1966 . Section One ; Some 
Characte rls tics of Children In the Head Start Program 
(Final Report) 

This report describes a normative study carried out 
by Educational Testing Service In the summer of 1966 as 
part of a large scale research program undertaken by ETS 
for the Office of Economic Opportunity. The normative study 
was designed to provide descriptive Information on children 
who have participated In Head Start programs. 

Two types of samples were drawn, one based on Census 
Bureau Information (N = 589) and one drawn from a stratified 
sample selected by ETS (N = 445). Instruments used were 
the Stanford-Blnet (L-M, Short Form), the Caldwell Pre- 
school Inventory, and the Project Head Start Inventory. 

The report presents data on the test results (Including 
Item omissions, distributions, and reliability estimates). 
The reader Is asked to exercise care In drawing Inferences 
from the data presented. 



R. H. Williams and Elizabeth Stewart 
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Project Heed Start S«"— 1966 . Section Two i Facilities 
and Resources of the Head Start Centers (Final Report) 

This report presents data collected in the Sunsier 
1966 Project Head Start program, by Educational Testing 
Service for the Project Head Start Research and Evaluation 
Division of the Office of Economic Opportunity. 

The "Center Facilities and Resources Inventory" (pro- 
vided by the Project Head Start Research and Evaluation 
Division) was sent to 630 Head Start Centers. Returns 
were received from 326 centers. 

Data are presented in the following areas: number 

and kinds of Center staff; children in the Centers: ages, 

ethnic background, behaviors; facilities and resources 
of Centers; and orientation and goals of Head Start programs. 

The Appendix contains a reproduction of the Center 
Facilities and Resource Inventory. 

J. L. Boyd 



Project Head Start Summer 1966 . Section Three : Pupils 

and Programs (Final Report) 

This report presents a result of an investigation of 
the relationships between pupil and program characteristics 
and pupil outcomes on certain prescribed Instruments and 
on other newly developed Instruments, carried out by 
Educational Testing Service for the Project Head Start 
Research and Evaluation Division of the Office of Economic 
Opportunity . 

The report presents answers to the following questions 
relevant to Head Start classes in the summer of 1966: 

Vlhere were Head Start children at the beginning? At the 
end? Were changes observed in the children? What are the 
reasonable alternative sources of these changes? Did 
Head Start classes differ from each other, and in what 
ways? Was there a relation between specific differences 
and amount of change? 

Information collected in the study "indicates that 
positive, consistent and developmentally desirable effects 
were observed during the summer 1966 program." 

The Information Collection Forms and Field Director's 
Manual used are reproduced in the Appendix. 



6. Temp and Scarvla B. Anderson 



24 



EVALUATION OF 

PROGRAMS 

(continued) 



Untangling the Tangled Web of Education 

Papers from a symposium sponsored In 1968 by the National 
Council on Measurement In Education, In conjunction with the 
annual conferences of the Educational Records Bureau and Educa- 
tional Testing Service. The symposium, dealing with research and 
measurement considerations related to assessing children's 
development In Interaction with school, family, and community 
Influences, covered the following topics: Early Schooling: 

What Is It All About?; The Family and Community t What Are Their 
Roles In the Educational Process?; The Child: His Cognitive, 
Person-Social, and Physical Development — A False Trlchoton^? ; 
How Are Measurement Strategies Related to Models of Human 
Development? ; C 2 ui You Do Real Research In the Real World? ; The 
ETS-OEO Longitudinal Study of Disadvantaged Children; and The 
Scientific and Social Slgnlflc 2 Uice of the Longitudinal Study 
of Dlsadv 2 uitaged Children. Copies are available from the 
Office of Information Services, ETS. 



On Evaluating Title 1 Programs 

Proceedings of a workshop on the evaluation of Title I 
programs conducted by Educational Testing Service In 1966. 
Papers are presented under two main topics: Educational Objec- 

tives and the Role of E^^aluatlon, and Developing Evaluation 
Instruments. Excerpts from discussions by participants are 
Included at the end of each main topic. 

Copies may be ordered from the Office of Information 
Services , ETS , 



Curriculum Innovations and Evaluation 

Papers presented at the 1968 ASCD Pre— Conference Seminar on 
Curriculum Innovations and Evaluation, held at Educational 
Testing Service, discuss the role of evaluation, approaches to 
the definition of objectives, uses and limitations of available 
Instruments, procedures for analyzing test results, principles 
and problems of research design, and Illustrations of the 
development and conduct of three types of curriculum evaluation 
projects. 

Copies are available from the Office of Information 
Services, ETS. 
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Biographical Information Blank as a Predict or of Student 
Achievement and Vocational Choice : A Review 

Although this review Is not directly concerned with the 
disadvantaged. It has been Included because of the Implications 
of the use of the biographical Information blank In studies of 
the disadvantaged. 

This paper reviews literature dealing with use of the 
biographical Information blank (BIB) among student populations 
from the late 1930 *s through 1966. The studies are considered 
the four criterion areas of scholastic grades , creative 
achievement, persistence In school, and vocational or curriculum 
choice. 

To date, the BIB has been of little practical value In 
academic prediction. The potential advantages of the BIB are 
cited, however, as well as suggestions for future research. 

N. E. Freeberg* 

RB 66-51 

Parental Influence on Cognitive Development In Early 
Childhood : A Review 

This review summarizes literature dealing with child- 
rearing practices that Influence cognitive development. The 
three major aspects of the literature considered are: (1) 

studies that attempt to relate measures of the child’s 
cognitive ability to particular parental rearing practices, 

(2) rearing practices characteristic of various social class 
levels as related to cognitive performance of children from 
those social classes, and (3) experimental attempts to enhance 
cognitive skills In the very young child by specialized training 
techniques . 



N. E. Freeberg and D. T. Payne* 

(See also: RM 66-1) 

Socio-economic Status and Predictive Test Scores 

A review of studies by Brigham, Davis, Lewis, Stroud, 
Conrad, and Robbins on the relation of socioeconomic status 
to scholastic achievement and Intelligence. Turnbull emphasizes 
the Importance of criteria in the Investigation of this problem; 
that Is, It Is essential to define the criterion to which a 
test Is Intended to relate, and then to justify Inter— group 
equality or Inequality of test scores on the basis of Its effect 
on prediction. . . .An equality of test scores that would signify 
fairness of measurement for one criterion on which cultural 
groups performed alike would signify unfairness for another 
criterion on which group performance differed." 

W. W. Turnbull 
RM 51-9 ' 
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Survey of the Literature Relating to the Effects of Cultural 
Background on Aptitude Test Scores 

This survey of the literature is divided into the 
following topics: Test Scores and Cultural Influence; 

Culturally Unbiased Tests: Some Methodological Shortcomings 

of Comparison Studies; Racial Comparisons; Occupational Group 
Comparisons; Rural-Urban Comparisons; School Comparisons; 
Socioeconomic Class Comparisons. Previous Reviews of the 
Literature and discussions of Aptitude, Cultural Influence, 
and Cultural Indices are Included. Each topic carries a summary. 
Bibliography. 



C. M. Lucas 

RB 53-13 

Testing of Culturally Different Groups 

The purpose of this study was to summarize research findings 
on testing of Individuals from groups, particularly Negroes, 
which differ in some respects from the majority culture in the 
United States. Particular attention was paid to factors which 
appeared to cause or be associated with changes in test scores. 
The effectiveness of tests in predicting performance of Negroes 
in Integrated and segregated colleges was evaluated. The 
summary also Includes a suggested program of research, designed 
to fill gaps in present knowledge, with particular attention 
focused on selection for college and remedying educational 
deficits at the college level. 

J. Campbell 

RDR 63-4, No. 14; RB 64-34 



On the Structure of Self-Esteem : Comments on Ziller *s Formulation 

Conceptual Issues In Zlller's research on social self-esteem 
(Zlller, Hagey, Smith, and Long, 1969) were reviewed in the light 
of recent theoretical and empirical work. Three limitations of 
the social self-esteem (SSE) measure and its supporting rationale 
were noted: 1) failure to distinguish between level and source of 

self-esteem; 2) Implicit "masculine" bias, and 3) Implicit cultural 
bias. These considerations would appear to explain the "negative" 
findings reported by Zlller et al. and suggest modifications of the 
SSE formulation which could extend Its generality. 



Rae Carlson 
RB 69-38 
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The Contribution of Clusters of Biographical Data to Prediction 

Although this study did not foctis directly on the 
disadvantaged, it has been Included because it Indicates that 
certain types of analyses of background factors might yield 
Information pertinent to low socioeconomic and minority groups. 

This study cluster analyzed the 169 Items of the ETS Growth 
Study seventh-grade form of the Background and Experience 
Questionnaire (BEQ) to develop homogeneous scales In order to 
determine what dimensions of personal background predict 
performance on aptitude and achievement measures, whether the 
background cluster scores add to the prediction of final 
performance that can be obtained from Initial test scores on the 
same test variables, and whether background cluster scores add 
to the accuracy of prediction that can be obtained from a 
comprehensive battery of initial cognitive measures. A random 
S£unple of 300 male and 300 female seventh-graders was selected 
from students who entered the ETS Growth Study as 5th graders 
in 1961 and were retested in 1964 and 1965, and their responses 
to the BEQ yielded 10 clusters for the male sample and 9 for the 
female sample. 

A stepwise multiple regression analysis was performed using 
STEP, SCAT, and the BEQ scores. The results indicate that when 
used alone, BEQ cluster scores show moderately high prediction 
of subsequent test performance. Statistically significant but 
quite small Increases to prediction were found when BEQ clusters 
were used In combination with measures of initial status. BEQ 
clusters added practically nothing to the prediction, however, 
when used with a battery of earlier test measures. 

J. A. Connolly 
RB 68-40 

The Interpretation of Law School Admission Test Scores for 
Culturally Deprived and Non-White Candidates 

An Ad Hoc Committee of the Executive Council of the Law 
School Admission Test Council was appointed to examine background 
factors particularly associated with cultural deprivation In 
relation to LSAT scores and academic achievement and to study 
the law school performance of white and non-white students. A 
questionnaire which was developed for the purpose of identifying 
students who were culturally deprived relative to their fellow 
students was administered in 16 law schools. 

None of the differences between the "culturally deprived" 
and "non-culturally deprived" groups were found to be significant 
when pre-law and LSAT scores were used as combined predictors 
or when LSAT was used as a separate predictor. Among 18 
statistical tests made using the pre-law record as a predictor. 
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three statistically different results were obtained: In one 

school, the "culturally deprived" group earned slgnflcantly 
higher grades relative to prediction, and for two schools, 
they earned significantly lower grades than predicted. 

A general conclusion was that the principles applicable to 
the use of LSAT scores with respect to students considered to be 
"culturally deprived" relative to their fellow students are 
those applicable to the total group. 

The number of non-white students was too small to permit a 
statistically conclusive examination of the prediction of their 
performance. 



Unpublished report of Ad Hoc Committee 
on Student Background Factors to 
the Executive Committee of the Law 
School Admission Test Council, June 1966 

Interpreting the Academic Records of Disadvantaged Students 

Problems discussed In this paper are: defining "disadvan- 

taged;" examining students' environment for factors which might 
enable them to adapt to demands of higher education; validity of 
test scores for the disadvantaged (l.e., "bias" and three possible 
definitions) ; whether background Information contributes to 
prediction of high school achievement beyond prediction provided 
by objective tests. 

Some of the conclusions were as follows: Test scores 

carried most of the weight In prediction, with little evidence 
of prediction from SES factors. For high school rank-ln-class, 
STEP Writing was best single predictor In high college-sending 
schools; SCAT Quantitative was best single predictor for students 
In low college-sending schools. 

T. L. Hilton 

(A presentation at a symposium on 
^ "Admitting Disadvantaged Students," State 

University of New York, Syracuse, 1965. 



Personal Background . Experience . and School Achievement : An 

Investigation of the Contribution of the Q uest ionnaire Data 
to Academic Prediction 

Data on 1,206 boys graduated from academic high school 
programs In 1961, Including SCAT and STEP scores and scores on 
a Test of General Information (TGI) were examined In relation 
to responses on a questionnaire concerning the students' 
experience during the previous year. Scales were created from 
unit weighting of subsets of 169 questionnaire Items to measure 
variables such as Amount of Low Level TV Watching and Level of 
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Occupational Plans. Criteria examined were scores from the 
Preliminary Scholastic Aptitude Test (PSAT) , College Board 
English Composition and American History Tests, and student's 
rank In graduating class. 

Biographical Information alone predicted objective test 
criteria and rank In class moderately well, but when used 
with objective test scores, the test scores accounted for 
practically all the variance In the criteria. "The ...data 
suggest that the art of constructing objective tests Is well 
ahead of the art of developing biographical Inventories." 

T. L. Hilton and A. E. tfyers* 

CEEB RDR, 65-6, No. 7; RB 66-5 

Predicting Success In Law School With Moderators 



Admissions committees frequently feel that scores on 
selection Instruments should not be Interpreted the same way 
for all applicants. A given selection Instrument may not yield 
equally accurate predictions for different kinds of applicants. 
This study was made to determine whether student background 
characteristics could act as moderators leading to the 
differential prediction of first-year grades In law school. 

The study Involved law school students, with first-year law 
school grade-point averages as the criterion; undergraduate 
grade-point average, the Law School Admission Test, and French's 
Insightful Reasoning Test as predictors; and, as moderator 
variables, the students' responses to a five-item questionnaire 
on age, college major, student's feeling about adequacy of his 
college preparation for law school, father's occupation, and 
time of decision to attend law school. 

It was found that background factors (e.g., father's 
occupation) did not add to validity, and that the value of 
background factors may lie In their effectiveness as moderators 
rather than In their utility as predictors. The multiple 
moderator system, however. Is useful for admissions officers In 
1) finding out for which types of candidates one can or cannot 
make accurate predictions, and 2) Identifying valid predictors 
for different kinds of law students. 

S. P. Klein, D. A. Rock, and 
F. R. Evans* 

RM 67-24 

The Relationship Between College Grades and Aptitude Test 
Scores for Different Socioeconomic Groups 

A study to compare the relationship between college grades 
and aptitude test scores for students of high, middle, and low 
socioeconomic status. Subjects were about 1700 male students 
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In seven colleges who took the Selective Service College 
Qualification Test (SSCQT) In 1951. Three socioeconomic 
groups were defined on the basis of responses to a questionnaire 
containing Items on Income and occupation of family head. 

Scholastic Aptitude Test scores predicted grades equally 
well and neither overpredicted nor underpredicted for all 
socioeconomic classes among college students, although It was 
noted that the population sample contained a very Inadequate 
^®P^®sentatlon of the lowest socioeconomic levels. 

D. G. Schultz 
RB 53-18 

Concurrent Validity of PM and S^ for Bilingual Dade 
County High School Volunteers 

Tae Scholastic Aptitude Test (SAT) and the Prueba de 
Aptltud Academlca, examination In Spanish which assesses verbal 
and mathematical aptitude, as does the SAT ~ were administered 
to 140 bilingual junior and senior volunteers In the Dade 
County High School system. In addition, a Student Questionnaire 
was administered to determine orientation to the Spanish and 

English languages and to obtain an Indication of socioeconomic 
status. 

The purpose of the study was 1) to develop and compare 
PAA-based and SAT-based equations for predicting grade point 
average (GPA), and 2) to determine whether measures of the 
student s orientation toward the English or Spanish language, 
or a measure of his socioeconomic status could be used together 
with his SAT and PAA scores to gain Improved concurrent validity 
against high school GPA as the criterion. 

When formulas for predicting GPA based on PAA and SAT were 
compared. It was noted that If 200 points were subtracted from 
the PAA variables , the resulting equations would be nearly iden- 
tical though the validities were not quite the same. This does 
not establish that PAA less 200 can be considered a substitute 
for SAT In general, but It lends more support to this practice 
than existed previously. 

Another finding was that students who appeared to be Spanlsh- 
language oriented tended to do relatively better on the PAA, 
although when the overall validities of the regression systems 
with varying weights for PAA and SAT were compared, they were 
found to be not greatly different. In fact, the verbal predictors 
did not aid much In the prediction; SAT-M alone was about as 
good a predictor as anything that was tried. Including the more 
complex systems Involving language orientation or socioeconomic 
status along with PAA and SAT. 

R. F. Boldt 

RDR-68-9, No. 3; SR-69-31 




Prediction of Performance for Negro and White Medical 
Technicians t Ethnic Membershi p as a Moderator of 
Superior *s Ratings 

This report documents findings derived from an attempt 
at criterion development for a larger study on fairness 
in selection testing. (See PR 69-4, page 31 , and PR 

69-6, page 32 ,) xhe criterion of supervisor's performance 
ratings was examined for evidence of whether the ethnic 
group membership of both the rater and the ratee Influenced 
the appraisal an individual received. 

Subjects were 168 Negro and 296 White Medical Tech- 
nicians at Veterans* Administration Hospitals. Each 
technician was rated by at least two superiors . Rating 
scales and a test of job knowledge were used. (See L. W. 
Pike, PR 69-4, page 31 ). 

Analyses of mean scores and ratings, correlations, and 
regression lines revealed that the race of both the person 
being rated and the person doing the rating has a note- 
worthy Influence on the evaluation received. 

An appropriate generalization from the study is that 
close attention must be paid to the nature of the criterion 
data in any prediction study. 

R. L. Flaugher, J. T. Campbell, and 
L. W. Pike 
PR 69-5 
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Prediction of Job Performance for Negro and White Medical 
Technicians ; Development of the Instrumentation 

Note: This report, together with PR 69-5, page 31 , and 

PR 69-6, page 32 , is from a study conducted jointly 
by Educational Testing Service Commission, funded by 
the Ford Foundation. 

The first phase of a study titled "Test and Job 
Performance of Various Subgroups Within Specified Occupa- 
tions" determined the feasibility of gathering data, on a 
large scale, relevant to problems of test validity for 
minority subgroups of the population. 

This report describes the development of the data 
collection Instruments used in the study, which was 
carried out on Medical Technicians . Aptitude measures 
consisted of eight tests drawn from the Kit of Reference 
Tests for Cognitive Factors, plus a Fine Finger Dexterity 
test developed by the United States Civil Service 
Commission. Criterion measures Included a Job 
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Rationale for the selection of tests from the Kit of 
Reference Tests and detailed Information on the construction 
of the criterion measures are provided. 

L. W. Pike 
PR 69-4 



Performance Appraisal Form for Medical-Technical-Technical 
Personnel, containing nine rating scales: a Job Knowledge Test; 

a Work Sample; and a personal history Questionnaire with a 
supplemental Task Information Checklist. 



Prediction of Job Performance for. Negro and White Medical 
Technicians . A Regression Analysis of Potential Test Bias : 
Predicting Knowledge Scores from an Aptitude Battery 

Will selected aptitude tests be biased when used to 
predict success In a particular occupation? The authors 
believe that the answer to this question depends on the selec- 
tion of an appropriate criterion. Using data from a study of 
Medical Technicians In Veteran's Administration hospitals, 
the authors present Information about the comparative strengths 
and weaknesses of the criteria of supervisory ratings, work 
sanqple, and a test of job knowledge for detecting such 
selection bias. Reviewing findings from a study by Flaugher 
et al. (PR 69-5, page 31 ), they suggest that neither super- 
visory ratings nor work-sample are as satisfactory validating 
criteria as a job knowledge test. 

The 63-ltem Job Knowledge Test and an eight-test aptitude 
battery were administered to 287 white and 168 Negro Medical 
Technicians. Means, variances, and Intercorrelatlons were 
calculated, and regression lines were determined for each 
aptitude against the Job Knowledge Test. 

Results showed that "the usual hypothesis of cultural 
bias Is not tenable. In that those Individuals with supposedly 
the more favorable background are at a disadvantage on a 
majority of the tests to be used as selection devices." The 
results". . .not only fall to expose the type of bias which 
would ordinarily be predicted, but. . .present some evidence. . . 
of what might be called a reverse unfairness. To the authors' 
knowledge, no hypothesis or theory exists to explain this 
phenomenon." 



J. T. Campbell, L. W. Pike, and 
R. L. Flaugher 
PR 69-6 
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PREDICTION The Use of Multiple Moderators In Academic Prediction 

(continued) 

The present study examined whether several student 
background characteristics (e.g., father’s occupation) could 
act as moderators leading to the differential prediction of 
law school grades. A new procedure based on a modification 
of Ward's hierarchical grouping technique was used for this 
purpose on the validation sample (n*201) . This resulted In a 
four— group solution using five moderators. Separate regression 
equations were constructed for each group and applied to 
corresponding groups In the cross-validation sample (n=201) . 

The results of these analyses Indicated that the new multiple 
moderator technique could reliably Identify groups that 
differed In predictability. 

S. P. Klein, D. A. Rock, and F. R. Evans* 
RB 67-50 
RM 67-24 



The Wide Range Validity of Certain New Aptitude Tests 

An earlier study revealed that adding certain types of 
new material to the Scholastic Aptitude Test (SAT) would 
significantly Improve Its validity for predicting the college 
performance of students of high ability. The purpose of this 
study was to see whether SAT validity might be Improved for 
students of only average ability. Nine colleges, with 9,500 
freshman students, participated. 

Only slight Increases In predictive validity were obtained. 
However, the results do suggest that certain new types of questions 
may Increase the predictive power of the mathematical section 
of the SAT. Another Important finding was that none of the 
recommended adjustments would Inject any discriminatory bias 
Into a test that has been carefully screened for bias. 

R. L. Flaugher and D. A. Rock* 

RB 66-54 
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RACIAL Differences in Selected Attitudes and College Orientations 

ATTITUDES between Black Students Attending Traditionally Negro and 

Traditionally White Institutions 

The purpose of this paper was to determine what 
educationally relevant differences exist, if any, between 
cwo groups of black collegians — those who enter tradi- 
tionally Negro colleges and those who enter Integrated 
ones. On the basis of data from the College Student 
Questionnaires (Part 1) and the Scholastic Aptitude Test 
(Verbal), black students entering Integrated Institutions 
were found to have higher SAT-V scores, to be more in- 
dependent, liberal and concerned with social injustice, 
and to aspire to more years of formal education. Many of 
the differences between the two groups, however, were 
found to be highly correlated with SAT scores. Thus, it 
would appear that to the extent Integrated institutions 
are attracting the higher ability (as measured by the SAT) 
Negro students, they are also attracting those with a 
quite different set of attitudes, background characteristics, 
and orientations toward college. 

R. T. Hartnett 
RB 69-48 



Coins in the Kettle ; Models and Donation Behavior 

"Coins in the Kettle" included experiments designed 
to gauge the "helping" behavior of Christmas shoppers in terms 
of their contributions to the Salvation Army workers* kettle 
In Princeton and In Trenton, N. J., when a "model contributor" 
was present and when no "model" was involved; and to assess the 
Impact of the solicitor's race on donation behavior, by having 
one of the Salvation Army kettles manned by a Negro and the 
other by a Caucasian. 

The presence of helping models significantly Increased helping 
behavior. The race of the Salvation Army solicitor did not 
affect the percentage of donors willing to contribute money when 
a model was present, but did affect donation behavior when no 
model was present. In that fewer donations were made when the 
solicitor was a Negro. 

J. H. Bryan* 

RB 67-10 
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A Distance Measure of Racial Attitudes In Primary Grade 
Children ; An Exploratory Study 

A nonverbal distance measure was used to assess 
Interpersonal racial attitudes among primary grade children 
In segregated and nonsegregated schools. Subjects pasted 
pictures representing themselves (self-figures) on a series 
of pages, each of which contained a fixed or "target" figure, 
such as Negro and white teachers, peers, and a school. 
Subjects could paste their self-figures In whatever position 
and at whatever distance they chose from the target figure. 

White subjects placed their self-figures farther from 
Negro than from white target figures; Negro subjects placed 
them equally close to white and Negro target figures. 

Compared with segregated subjects. Integrated subjects 
placed the self-figure closer to target figures of the 
opposite race and closer to the school. 

Sandra Cohen Koslln, Marianne Amarel, 
and Nancy Ames* 

RB 69-11 

The Effect of Race on Peer Evaluation and Preference In 
Primary Grade Children ; An Exploratory Study 

Attitudes of students In primary grades were explored 
with regard to the learning ability, school behavior, and 
social desirability of white and Negro children as classmates. 
In Individual Interviews, students at an all-white, an all- 
Negro, and an Integrated school were shown pairs of pictures 
of classroom events taking place In predominantly white and 
predominantly Negro classes, and were asked to make judgments 
about the academic achievement, niceness, school behavior, 
friendliness, and general desirability of the children In 
the pictures. 

White subjects showed a striking preference for 
predominantly white classes. Negro subjects were less 
consistent: some preferred white classrooms; some Negro 

classrooms; and others showed no preference at all. 

Sandra Cohen Koslln, Marianne Ai^rel, 
and Nancy Ames 
RB 69-12 



36 



SCHOOL 

CHARACTERISTICS 







Conmunlty Socioeconomic Status , Aptitude , and School Achievement 

This is an analysis of Growth Study data, 1961 through 1963, 
consisting of results on SCAT Level 5 and STEP Level 4 Math, 
Science, Social Studies, Reading, Listening and Writing, grades 
5 and 7; plus questionnaires sent to principals of participating 
schools, requesting information on school characteristics. Schools 
were classified on a basis of SES using questions 45-49 of the 
latter. Each student was classified as high- or low-ability by 
dichotomizing SCAT scores. Four groups emerged: 1) hlgh-ablllty 

students in high SES schools; 2) high ability students In low 
SES schools; 3) low ability students in high SES schools; and 
4) low ability students in low SES schools. 

The findings on SES indicated that dichotomizing schools on 
SES and students on SCAT does not consistently Increase accuracy 
of prediction for all STEP tests, but dichotomizing just schools 
on SES does consistently Increase such predictions. Ability 
emerges as a more important determinant of achievement than SES. 
Students tend to become more heterogeneous In achievement as they 
receive more education. 



T. L. Hilton and L. A. Plngel 
(A presentation at the National Council 
on Measurement in Education Meeting, 
Chicago, 1965.) 

A Study of Secondary School Characteristics as Related to 
Test Scores 

A study made in 1956 to determine what characteristics of 
public secondary schools were most closely related to the 
performances of their students on academic aptitude and achieve- 
ment tests. Questionnaires were sent to public high school 
principals to elicit information about their schools, education 
and occupation of parents, and community characteristics. 

Ninth- and 12th-grade students were administered a four-part 
aptitude £ind a three-part achievement test. 

Concomitant variation (e.g., parents of high Intellectual 
ability deciding to live in areas where schools are good) 
accounted for a high portion of correlations between school- 
parent- community characteristics and test scores. Aptitude 
scores are probably also affected by children's educational 
experiences. Support, instruction, and attitude toward 
learning make a difference in school achievement, as shown In 
slze-of-class variable and the time teachers have for individual 
instruction, and the percent-to-college as a school attitude 
variable . 



W. G. Mollenkopf and S. D. Melville 
RB 56-6 



An Investigation of Item Bias 



A study of the variation of Preliminary Scholastic 
Aptitude Test (PSAT) item scores in different racial and 
socioeconomic (SES) groups, to determine tdiether items in the 
test are biased (bias being defined as item x group interaction.) 
A sample was drawn from Growth Study participants: Group I 

(1961), 636 twelfth-graders and Group II (1963), 724 twelfth 
graders , using all available Negro students and a random sample 
of an equal number of white students. SES was defined from a 
score on the Background and Experience Questionnaire given in 
the Growth Study, using items concerned with father’s occupation 
father's and mother's education, and the House-Home Index (Kerr 
Remmers 1942) . 

In four separate analyses, the Item x Race and Item x SES 
Within Race Interactions contributed minimal percentages of 
the total variance of an observation. From the bivariate plots 
of sums of item scores, there were few Items producing an 
uncommon discrepancy between the performance of Negro and white 
students. Given the stated definition of bias, it was concluded 
that the PSAT for practical purposes is not biased for the 
groups studied. 



T. Anne Cleary and T. L. Hilton* 
CEEB RDR 65-6, No. 12; RB 66-17 



Test Bias : VaMdity of the Scholastic Aptitude Test for Negro 

and White Students in Integrated Colleges 

To determine whether the Scholastic Aptitude Test is biased 
as a predictor, a study was made of the regression of college 
grades on the SAT for Negro and white students in Integrated 
colleges. College grade-point average was the criterion, with 
high school grade average and rank In class Included wherever 
possible. Samples were drawn from two eastern state-supported 
or subsidized colleges and one , state-supported college in the 
southwest . 

Results showed little evidence that the SAT is biased as 
a predictor of college grades, although the regression lines 
for white and Negro students were slgnflcantly different in 
the southwestern college. When high school grades or rank in 
class was used in addition to the SAT as a predictor, the degree 
of positive bias for Negro students Increased. 

T. Anne Cleary* 

RB 66-31 
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The Effect of Special Ins truct ion Upon Test Perf of 
High School Students In Tennessee ' 

To determine whether special Instruction would benefit 
students who had received less adequate instruction and who 
were less able academically than students in previous coaching 
studies, special linear programmed materials in verbal and 
mathematical areas were administered to llth-grade students 
in 18 predominantly Negro high schools. Alternate forms of 
the Preliminary Scholastic Aptitude Test were used for 
evaluation. 

Although there were gains in the experimental group, 
these were more than offset by reductions in post test scores 
In control groups. The outcome of the study was generally 
negative, as has been true of most previous studies on 
coaching . 



S. 0. Roberts and D. Oppenhelm 
CEEB RDR 66-7, No. 1; RB 66-36 



Ibept and Test Analysis Results for Can didates Tested at the 
Free LSAT Administration of February 1967 ~ 

In order to encourage more Negroes to consider a career in 
law, a special) free administration of LSAT was conducted in 
February 1967 at a number of colleges having predominantly 
Negro enrollment. A sufficiently large number of candidates 
participated to warrant a study of the statistical characteristics 
of the items and of. the separately-timed sections of LSAT, 

Writing Ability, and General Background for this group. These 
®^^^^®bical results were compared with comparable data based on 
a cross-section sample of regular candidates and on a 
stratified sample of regular candidates having substantially the 
same level of scores on each test as the group attending the 
free administration. 

Poorer performance by the free administration candidates 
than by the regular candidates characterized substantially all 
the items and the various test sections. The reliability of 
LSAT, Writing Ability and General Background scores was 
approximately the same for the free administration sample and 
the two comparison samples. The free administration candidates 
tended to answer fewer questions than the regular administration 
candidates, which works to their disadvantage. 

Barbara Pitcher and W. B. Schrader 
Unpublished report to the LSAT 
* Council, May 1968 
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Special Item Analysis of thd Admission Test for Graduate Study 
In Business for Negroes 

The Consortium for Graduate Study In Business for Negroes 
was established In 1966 to coordinate programs concerned with 
Increased participation of Negroes In advanced business education. 
Negro students Interested In such education, who take the 
Admission Test for Graduate Study In Business as part of this 
program are referred to as Consortium candidates. Distribution 
of Consortium candidates' scores on ATGSB was matched with 
that of non-Consortlum candidates (all U.S.). 

A sample of non-Consortlum candidates was selected to 
match Consortium candidates on total score. It was found that 
the two groups differed significantly on verbal and quantitative 
scores, with the Consortium groups scoring higher on verbal. Two 
additional samples were drawn: one to match the Consortium group 

on verbal and the other on quantitative. 

The test as a whole seemed to be equally reliable for both 
Consortium and non-'Consortlum candidates. There was a significant 
difference In the percent correct between the two groups on 12 
verbal and 14 quantitative Items and there was a remarkable 
similarity In choices of the same wrong answers by both groups. 
Among quantitative Items, those Involving percentages were 
significantly more difficult for Consortium candidates. 



W. Cowell 

ATGSB Special Report (In Press 1969) 
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THEORIES, 
METHODS, AND 
TECHNIQUES 



Although some reports In this category are not directly 
concerned with the disadvantaged, they have been Included 
because they deal with theories, methods, and techniques 
which might be tiseful In further Investigations of the 
characteristics of the disadvantaged. 



Assessment of Disadvantaged Adolescents ; A Different Approach 
to Research and Evaluation Measures 

Limitations of available formal Instruments for use with 
culturally deprived subjects are described. In an attempt to 
overcome some of the claimed defects of some of such measures 
(cognitive and noncognltlve) , a battery of measures was 
specifically constructed, "tailored” for use with disadvantaged 
groups. In the areas of Job Knowledge, Vocational Aspirations, 
Vocational Plans, Interest In Vocational Tasks, Attitude Toward 
Authority, Self-Esteem, Deferred Gratification, Job Seeking 
and Job Holding Skills, Motivation for Vocational Achievement, 
and Practical Reasoning. 

The measures, designed with minimal verbal content at 
approximately fifth-grade level, with simple format (one Item 
per page), appropriate pictorial Information, and short test 
length (not more than 30 Items In a booklet), were administered 
to 256 male and female high school dropouts enrolled In a 
federally funded youth-work training program. Included In 
the measures was a 15-1 tern Enrollee Rating Scale with questions 
to answer on his feelings about the program. 

Item characteristics, levels of Internal consistency, 
external validity based on rating criteria, construct validity 
based on factor analysis, and the logic of a number of research 
findings with the Individual measures, presented a pattern of 
results reasonable enough to warrant continued research 
application and development of these types of measures for the 
purpose Indicated; however, the measures do not yet possess a 
formal status that would justify broader applications, e.g. 
as guidance, placement, or selection Instruments. 

N. E. Freeberg 
RB 69-41 
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THEORIES , Disadvantaged Children and Their First School Experience 

METHODS , AND 

TECHNIQUES A. report covering the period February 196 8- January 1969 of 

(continued) ETS plans for the Longitudinal Study of Disadvantaged Children, 

under OEO auspices. Contains an Overview of the Study, 
summarizing principal features of the study design, an outline 
of Strategy and Tactics In Conducting Research with the 
Disadvantaged, and chapters dealing with approaches to measure- 
ment and evaluation In the following areas: Children's Cognitive 

and Perceptual Development; Children's Personal and Social 
Development; Children's Physical Health and Nutritional Status; 
The Impact of the Family; The Impact of the Classroom; The 
Impact of the Teacher; The Impact of the School; The Impact of 
the Community; The Impact of the Tester. Measurement strategies 
are summarized, and procedures for sampling, measurement plans, 
field operations, testing centers, and scheduling are presented. 
Final chapters deal with Systems Design and Control, and 
Analysis . A separate appendix Is related to measures . 

Scarvla Anderson, A. Beaton, W. Emmerich, 
and S. Messlck 
PR 68-4 



An Exploratory Study of Student Attitudes In the Primary Grades 

The purpose of this exploratory study was to Investigate 
the feasibility of using a structured projective Interview to 
study the attitudes of primary school children toward school 
and toward teachers. A series of sketches depicting teacher- 
child or child-child Interaction In the classroom were shown to 
112 first- and second-graders In 7 schools. The children were 
asked to tell what led to the pictured situation or what would 
happen next. 

The responses were judged Independently by judges In terms 
of overt content and underlying attitudes revealed by the 
responses. Items believed to be tapping aspects of the same 
underlying attitudes generally showed modest but slgnf leant 
correlation s with each anA M-rrt — 

believed to be unrelated. Responses to Items did not differ 
according to sex, grade, or socioeconomic background. However, 
children In an urban slum school gave a greater number of 
responses expressing violence than did children In other schools. 
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Children showed a high degree of responsiveness to the 
interview with virtually no refusals to respond, although 
skilled and patient interviewing was required in many cases 
to obtain complete protocols. 

Sandra R. Cohen 
RB 65-30 



A Feasibility Study of National Assessment : Some Implications 

for Test Construction and Administration 

A study in connection with the variety of items in the 
National Assessment Item pool, designed to obtain information 
on 1) the number of possible formats that should be presented 
to any one individual; 2) the number of subject areas which should 
be represented in an "item package" (since "tests" were not to 
be given in specific subject matters, would variety of content 
confuse the student?) ; and 3) conditions under which assessment 
might be conducted, with emphasis on how well teachers would be 
able to administer items to groups of students. ’ 

Low-ability students in low-income areas in eight different 
communities in rural, suburban, and center-city locations were 
used for a case-study approach. Items were presented to groups; 
following group administration, selected students were administered 
items individually and asked to comment on them. 

Some findings: In general, students were not confused either 

by the number of subjects or variety of format. Only two item- 
types seemed to generate confusion. One type involved responses 
in sets; another type involved negative response (least , worst , 
smallest) . Some problems for those administering the tasks were 
revealed. 



Requests for copies should be addressed directly to the author 
at ETS, Princeton, New Jersey 08540 

G. G. Gordon 

Paper presented at NOME Convention 
Los Angeles, February 1969 
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Factor Stability of £ Student Biographical Information Blank 
Over Several Grade Levels 

A biographical Information blank (BIB), administered to 
students In grades 7, 9, and 11 In 1963 as part of the ETS 
Growth Study, was analyzed to determine the stability of Its 
factor structure. This was done by deriving a single average 
factor pattern matrix which was rotated to simple structure 
and against which factor matrices at each grade level could be 
compared. Of 11 factors extracted, all but three showed 
reasonably good stability. Those factors that showed the 
greatest change. In terms of their presence over the three 
grade levels, were "General Appreciation of School Courses," 
"Social Activities," and "High Level Literary Activities." 

N. E. Freeberg and D. A. Rock* 

RB 68-18 

A Method for Comparing the Performance of Different Groups on 
the Items In ^ Test 

The purpose of this study was to develop a method which 
separates the OV' ^rall differences In test performance of 
different subsamples of a general population from differences 
attributable to Individual questions, — a method which would 
provide an estimate of the Interaction of Items with subsamples. 
The two-factor analysis of variance design with repeated 
measures on one factor Is demonstrated to be appropriate to the 
problem If Item difficulties are first subjected to the arisen 
transformation. Application Is Illustrated using SAT candidates 
from 3 different subgroups of the total population — Negro 
candidates frpm the Southeast, candidates from small town and 
rural centers In Indiana and Illinois, and candidates from 
centers In the Bronx, New York. 

Carolyn Cardall and W. E. Coffman 
RB 64-61 

A Moderated Stepwise Prediction Sys tern 

Describes the development of a computer-based statistical 
system which Is general In nature but which could yield some 
tentative answers to questions concerning Interactions between 
groups of Individuals and their attributes when predicting some 
criterion of success. The technique Is multivariate; that Is, 
complex multiple Interactions may be handled simultaneously. 
Thus, given a sample of Individuals with multiple measures on 
each. Including a criterion of success, the system can be used 
to Isolate groups consisting of Individuals with common profiles 
on background or personality variables, which also yield optimal 
wlthln-group prediction systems. The system could also aid In 
finding out whether a person's profile based on multivariate 
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Information dealing with past experiences (e.e., background or 
demographic measures) can substantially affect the prediction 
of achievement criteria from normal aptitude measures. Thus 
one could find and describe types of individuals who 1) either 
require different attributes for success or 2) have differing 
patterns of the same attributes yet achieve the same level of 
success. Methodology is described, along with Grouping on 
Profiles, Within-Class Regressions, and Intercept Method. 

(For an application of this method, see The Use of Multiple 
Moderators in Academic Prediction , page 33 ; A Multiple 
Moderator Approach to the Identification of Over- and Under- 
achievers, page 44 ; and The Identificat ion and Utilization 
of Moderator Effects in Prediction Systems, page 7 ) 

D. A. Rock and J. L. Barone* 

RM 67-11 

The Identification and Utilization of Moderator Effects in 
Prediction Sys terns 



Traditional linear multiple predxction models assume that 
a given set of tests predicts with equal accuracy for all 
members of a heterogeneous pxpulation. Evidence suggests that 
this assumption is sometimes false, that tests may have 
differential validity for homogeneous subgroups of larger 
populations. Variables defining such subgroups for which there 
is differential prediction are called moderators . That is , 
these variables moderate the nature of the relationship between 
predictor (s) and criterion. Examples are cited using background 
and demographic variables to define subgroups for which senior 
class rank is predicted with differential accuracy by subtests 
of a standard achievement test. , For subgroups defined by other 
moderators, prediction was more accurate for some of the sub- 
groups but less accurate for others. An implication of this 
latter finding is that counselors or personnel administrators 
must be aware that they can predict performance accurately for 
some subgroups but that for other subgroups they should withhold 
judgment pending further information. 

A general classxfxcatxon procedure, following a development 
by Tatsuoka, is suggested as a means of combining information 
from both moderators and predictors into one probability 
statement. 



D. A. Rock 
RB 69-32 

A Multiple Moderator Approach to the Identification of Over- 
and Underachievers 

This paper reports a study of over- and underachievement 
of college students, utilizing a computerized multiple moderator 
technique. The subjects were students at a large southwestern 
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university; the predictors were short forms of the SAT and high 
school rank; and the criterion was first-year grade average. 
Selected background variables were the potential moderators. 

The overachievers were characterized by having average 
abilities yet coming from backgrounds where the father was 
highly educated, while the underachievers were observed as 
having small- town origins and high Interest In extracurricular 
activities . 



R. L. Flaugher and D. A. Rock 
RB 69-26 




STUDIES IN PROGRESS 



BACKGROUND FACTORS 

Survey of the Academic Progress and Financial Status of Enrolled Students 

One of the main questions to be answered by this study Is that 
involving the financing patterns of students from various socioeconomic 
backgrounds . 



J. L. Bowman 



Measuring Financial Need 



The College Scholarship Service is examining the appropriateness of 
its procedures for measuring financial need of students from disadvantaged 
backgrounds. The study will examine the special problems of such students 
from the economic side and make recommendations for changes In current 
CSS procedures. 



J . L . Bowman 

COGNITIVE PROCESSES, DEVELOPMENT, AND FUNCTIONING 
Let’s Look at Children 



Earlier work on this project Is described on page 7 under Let’s 
Look at First Graders . 

During 1967-68, children from middle- and lower-class neighborhoods 
were observed and tested. The results of these studies of cognitive 
performance will be used as a basis for developing additional materials to 
help preschool and kindergarten teachers make more adequate classroom 
appraisals of children’s learning and abilities. 

E. A. Chittenden 



EVALUATION OF PROGRAMS 

Neighborhood Youth Corps 

Phase I of this project is reported on page 17 Phase II will be 
concerned with the development of criteria for the performance of enrollees 
in the Neighborhood Youth Corps. A study of background data as predictors 
for disadvantaged adolescents in a youth work training program will also 
be conducted. 



N. Freeberg 



Evaluating Title I Activities in the Chicago Public Schools 



A series of studies Is being initiated to determine the outcomes of 
programs initiated by the Chicago Board of Education under funding through 
E.S.E.A. , Title 1. Several papers and memoranda will arise from these 
studies during 1970. Previous work on this project is reported on page 18 . 

G. L. Marco 

Evaluation of Integration of Evanston District 65 Schools 

Evaluation of achievement outcomes of an integration program for the 
elementary schools in Evanston, Illinois, Is being conducted under 
subcontract from the Rockefeller Foundation. A final report Is scheduled 
for 1971. Previous work on this project is reported on page 17 . 

D. P. Norton and J. Hsla 

External Educational Accomplishment Audit for ESEA Title VIII (Dropout 
Prevention) Programs 

Educational Testing Service is conducting the external educational 
accomplishment audit for two Title VIII Dropout Prevention Programs: the 

Talent Identification and Development Program sponsored by The Dade County 
(Florida) Board of Public Instruction and the Potential Dropout Recognition 
and Prevention Program sponsored by The Chautauqua County (New York) Board 
of Cooperative Educational Services . 

The audit, which is required by the United States Office of Education 
for all Title VIII Programs, Is an external assessment of the process, 
product, and management of the program. Its objectives are to verify the 
results of local evaluations and to assess the appropriateness of program 
procedures. The primary methods of accomplishing the audit are on-site 
observations and examination of program records and reports. The results 
of the audit are reported to the USOE, the program management, and the 
general public. 



D. W. Seibel 

Children’s Television Workshop 

Special television programs designed to teach preschool children 
through enjoyable entertainment will be telecast by the Children’s 
Television Workshop (CTW) In a 26-week series of hour-long shows five days 
a week, beginning in November, 1969. Puppet shows, story readings, 
animated cartoons, and fast action films are some of the program activities 
that will be employed to Introduce educational concepts and learning 
skills. The aim of CTW "is to narrow the academic achievement gap between 
disadvantaged and middle-class children which appears early in school and 
increases in the higher grades." The project is being funded by government 
and private agencies. 

Consulting services are being provided by ETS to help CTW develop 
behavioral objectives and formative and summative research procedures. 
Weekly assessments of the show will be made, and the overall Impact of the 
show will be measured in terms of the objectives of the program. 



S. Ball 
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Evaluation of Verona Plan for Sharing Educational Opportunity 

A total of 38 educationally underprivileged youngsters from Newark, 
most first and second graders, have been attending classes In suburban 
Verona under the Verona School Board's Plan for Sharing Educational 
Opportunities. Each day they are bused to Verona, accompanied by a 
teacher whose task It Is to Involve herself in any special problems these 
children may encounter. 

ETS, In conjunction with the school boards of Verona and Neward, Is 
conducting a study of the outcomes of the program which will focus on 
student achievement In reading, arithmetic, and listening; student attitude 
changes toward racially mixed classes; parent attitudes; and operational 
problems encountered by teachers during the year. 

S. M. Zdep 



Follow-Through 



Educational Testing Service Is working with the Education Development 
Center to develop broad, more diverse, assessment approaches to evaluate 
the effects of Innovative educational efforts represented by Head Start 
and Follow Through programs. The educational goals of such programs will 
provide the frame work for designating the areas of assessment. This 
project represents a cooperative effort on the part of professional staff 
at both organizations who are concerned with problems In assessing young 
children. 



E. A. Chittenden 

A Study of the Practices. Development. Effect and Administration of the 
Educational Talent Search 



The Educational Talent Search Program of the Office of Education has, 
since 1966, funded about 100 projects. The primary aim of these projects 
is to "identify qualified youths of financial or cultural need with an 
exceptional potential for post-secondary educational training and encourage 
them to complete secondary school and undertake pose-secondary education." 
ETS has been awarded a contract by OE to gather data from project directors, 
students, and community leaders as to the effectiveness of the projects 
with particular emphases on Improving the program In future years. 

A. Carp and 6. Temp 
College Success and Low Test Scores 

Under a small grant from the U.S. Steel Corporation, three or four 
Institutions with seemingly Innovative and successful programs for high 
risk students will be Intensively studied. Efforts will be focused on 
objectively describing "input, treatments, and outputs" so that some 
Idea of the effectiveness of remedial programs can be determined. 



A. Carp 
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A Pilot Study of Minority Group Students at Highly Selective Institutions 

This study will be conducted to evaluate college performance of 
disadvantaged students who show academic potential and are given appropriate 
counseling and financial aid. It Is anticipated this research will yield 
Information on the difficulties the student has encountered and how he 
overcame them. 



W. S. Hall 



PREDICTION 

The North Carolina Study of Admission Policy and Practice 

As an Initial determination of potential racial bias In preadmissions 
Indices (notably SAT scores and high school grades) available to public 
college admissions officers In North Carolina , this study will examine the 
predictive validity of these Indices , as evidenced through the performance 
of 1968-69 freshmen. In each of the 16 public Institutions of the state. 
Analysis of covariance will be used to determine the significance of 
differences between the regressions for white vs. Negro students. Results 
will be utilized by the sponsoring North Carolina Board of Higher Education 
for framing a new statement of admissions policy. Later phases of the 
study will examine the origins, biographical characteristics, and adjustment 
to college of students who are In the racial minority for particular 
campuses (traditionally black or white), toward more effective guidance 
and educational treatment of minority students, and toward easing real and 
subtle barriers of access to higher education opportunity. 

J. A. Davis 



RACIAL ATTITUDES 

Some Applications of the Market Survey Technique for Institutions of Higher 
Education 



For a small college serving a relatively self-contained geographic area, 
a "market survey" of all high school seniors was conducted. A biographical 
Inventory was used to obtain Information on background, achievement In high 
school, post high school aspirations, and the Images held of area colleges. 
Among other analyses, the responses of Negro students are being contrasted 
with ttiose of white students, with preliminary results Indicating that 
(1) Negro students see some traditionally white colleges that proclaim 
receptivity to their applications to be hostile at worst, cool or uninterested 
at best, and (2) stereotyped perceptions of traditionally white colleges 
are more poorly defined and undifferentiated for the Negroes who aspire to 
enter college. 




J. A. Davis 
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Error-Choice 



This study of the measurement of attitudes toward Negroes uses a 
"disguised, structured" method of measuring attitudes — the error-choice 
technique — In which subjects are asked to respond to a series of apparently 
factual multiple-choice Items concerning desirable and undesirable 
characteristics. Attitude Is thus defined operationally as consistency In 
estimating the Incidence or degree of desirable and undesirable characteristics. 

The data will be analyzed to determine the dimensionality of the set 
of error-choice Items and to determine the relationship between a best 
subset of Items and the criterion Instruments. 

T . S . Barrows 



TEST CHARACTERISTICS 

Test Bias and the National Teachers Examination 



This study Is being conducted to Insure fairness to disadvantaged 
students taking the National Teachers Examination. Because large numbers 
of users In the South and In northern metropolitan areas are testing 
candidates from the population generally labeled as disadvantaged, the 
General Education section of the Common Examinations Is being studied for 
bias. Bias Is defined as the tendency of Items to be more difficult for 
one population than for another. If the subcultures from which teachers 
must be recruited can be Identified with care, the results of the analysis 
would Indicate clearly (and In what way) It may be possible to construct a 
test In this general area which will give teachers from all such subcultures 
an equal opportunity to earn a particular score on the test. 

D. M. Medley 

Test Blast Validity of the SAT for Negro and White Students In Integrated 
Colleges 



This study to be carried out In 1969 and 1970 Is essentially a repeat 
of the Cleary study repojrted on page 37 as RB-66-31. A sample of colleges 
using the SAT and admitting sizeable numbers of black students will be 
asked to provide test scores and criterion data. The Increased participation 
of blacks In higher education and possible changes In treatment of minority 
groups on campus makes It worthwhile to repeat this study In a sample of 
10 to 15 colleges. 



G. Temp and A. Carp 



TEST PERFORMANCE 

Student Test-Taking Behavior 

The purpose of this study is to: 1) obtain infromation about student 

test-taking behavior, 2) examine the relationships between teacher's 
evaluations and external observers' evaluations are influenced by factors 
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such as student socioeconomic level, class size, age and grade level, 
test performance, etc. 

The administration of the tests included the presentation of all 
test Items by paced tape. Among the comments offered by teachers and 
observers was one Indicating that the low SES students who had reading 

and language problems were helped by hearing, as well as seeing, the 
exercises . 



D. Seibel, Scarvla Anderson, and Arleen Barron 

THEORIES, METHODS, AND TECHNIQUES 
Social Stratification 



This study has two aims: a) to identify the dimensions of social 

stratification for both Negroes and whites, and the indexes that best 
reflect these dimensions, and b) to determine whether distinct social 
classes exist and, if so, whether they can be empirically differentiated. 
Although a vast amount of research has been devoted to the identification 
and development of objective indexes of social stratification, such as 
education and occupation, this research has been based almost entirely on 
samples of whites. 

A comprehensive, highly structured interview schedule was prepared to 
obtain information that directly reflects the major aspects of social 
stratification or important correlates of it. The schedule was used in 
household interviews with representative samples of Negro and white household 
heads in a typical northern city. 



L. J. Strieker 



Critical Issues in Research Related to Disadvantaged Children 



The proceedings of six seminars on critical issues in research related 
to disadvantaged children from birth through third grade will be contained 
in this report. The topics of the individual seminars and the seminar 
editors are: Motivation, Urie Bronfenbrenner ; The Teacher and Classroom 

Management, Martha Rashid; Head Start Populations, Myles Friedman; Health 
and Nutrition in Early Childhood, A. Frederick North, Jr.; Intervention 

in Family Life, Robert Hess; and The Teacher in Intervention Programs, 
Irving Sigel. 



NW. 



Copies will be availalbe from the Head Start Office, 1111 18th Street, 
Washington, D.C. by the late fall of 1969. 



Ruth Ekstrom 
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